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three major educational revolutions have been the 
advent of writing, the use of books as teaching tools, and the shift 
in educational responsibility from the home to the school. The fourth 
revolution^ which is based oh the use of electronic _ technology in 
teaching, began with programmed teaching machines,, individualized 
instruction, and the development of computer based education. This 
paper draws oh meta-ahalyses conducted at the University of Michigan 
and the University of Cbloradp to review what has been learned from 
the_ application of educatibhal evaluation arid research synthesis 
tools tothe area of iristruct ibrial techriblbgy. Eight separate 

meta-analyses arecbvered, represeritirig at least 500 separate 

studies. The results preserited indicate that progranmied instruct ion 

was mbre effective in recent studies than in earlier ones ; 

individualized systems produce about the same results as convent ional 
teaching at the elementary anci secondary levels, although at the 
college level they are strikingly effective; and computer based 
education has real potential as a tool for improving studeht 
achievement. Eight references and four ficures are appended. (LMM) 
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At least three major revblutibhs have swept over _ . 
education during the past 2000 years. The first occurred, 
many centuries ago when writing began.tb be_ used as a tool 
in teaching^ and learners were freed from the task of rote 
memorization of every point. _ A. second revolution came with 
the invention of printing and. the subsequent u$e of books as 
a teaching medium, Aud the third came when pirimary. 
responsibility for educating the young was shifted from the 
home to the school. 

We are now witnessing a fourth revolution in education, 
it is based on the use of .electronic technology in teaching, 
and it has already changed the look of our classrooms . No 
one is surprised nowadays to find young learners working on 
arithmetic or language exercises at computer stations in our 
schools. Nor is it unusual to find school children learning 

about literature, science^ or society from films and 

videotapes. Older students use language tapes in fore ign 
language courses^ and they use records and tapes in music 
appreciat ion classes. They analyze physical and social ^ data 
on computers / and they write reports using word-processing 
programs. 

This fourth revolution in teaching began, modest ly ^ 
enough nearly 30 years ago, and its progress has come. in 
three waves. The first wave of interest was generated by 
B. F. Skinrer. in his 1954 article "The Science of Learning 
and the Art. of Teaching ," Skinner _ argued that machines could 
teach more reliably and effectively than human teachers do, 
and he described programmed teaching machines of his own 
design that were already being used in instruction. The 

programmed machines .waited patiently for a learner's 

response, reinforced the response immediately, and stuck to 
a carefully crafted script. 

The second stage in this revolution was marked by the 
development of individualized instruct ion . Like programmed 
instruction, individualized teaching emphasi zed independent 
work, _ self-pacing^ and the achievement of mastery, but it 
used longer instructional units--of ten called learning 
activity packages or mqdules--and it gave learners more 
freedom to choose among different instruct ional means. 
Individually Prescr ibed Instruct ion , Project Plan, _ ^ 
individually Guided Education, and Keller's Personalized 
System of Instruct ion are probably the best known of the 
individualized systems developed during the 1950s. 

The third stage in the technological revblut ion was 
marked by the development of computer-based educat ion . 
Computers were first used in education to deliver programmed 
and individualized instruction, but in. recent years , they 
have been used to do more and more sophisticated teaching 
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jobs. Some educators are now arguing^ in fact, that 

students learn most from computers when they simply 

communicate with them in languages the computers understand. 

The first three _revolut ions in education ran their _ 
course without . help.frbm educational research. No one tried 
to measure their effects while these revolutions were in 
progress. No. one used statistical methods to predict or 
influence their outcomes. The revolutions just happened — 
without measurement, predict ion, or control. The fourth 
revolution is different. It is occurr ing at a t ime when we 
have tools f or _ evaluat ing specific programs and tools ^ for 
drawing general conclusions from a collect ion ^ of specific 
evaluat ions , The fourth revolution is occurr ing at a t ime 
when educa-^.ibnal measurement, prediction, and control are 
possible. 

The tools of educational evaluation, and research 
synthesis have already been applied to the. area of ^ 
instructional technology, and my purpose this ^morning is 
simply to review what we have learned. from this work. My 

account will draw heavily on meta-analyses carried out at 

the University of Mich igan, _but it will also cover work done 

at the University of Colorado, In all, my account will 

cover eight separate meta-analyses and at least 500 separate 
studies. To put some order. into my review, I will divide it 
into three parts. First, I'll talk about programmed 
instruction and its effects on learners. Then, i '11 move on 
to individualized systems and their effects . Finally , I'll 
look at the effects of computer-based education. 

Program med Inst rue t ion 

The first meta-analysis of findings in this_area_was 
carried out at the University of Colorado in 1977 by Susan 
Hartley. She located a total of 40 separate studies carried 

out between 1962 and 1974 in elementary and secondary 

mathematics classes. In the typical _ study , achievement _ test 
scores of programmed and conventional groups were virtually 

indistingul'JhlBie."" ' They differed by. only 0.11 standard 

deviations. The group mean was. at_the 50th percent ile for 
the conventional group and at the _5_4th percent i le for the 
programmed group. The. results of Hartley's analysis appear 
at the left of Figure 1. 



Insert Figure 1 about here 



Our research team at the University of Michigan also 
carried out two meta-analyses of findings on programmed 
instruction, one covering 48 studies at the secondary level 
and one covering 57 studies at the college level (G.-. 
L,_Kulik, Shwalb, & Kulik, 1982; J. Kulik, Cohen, & Ebelmg^ 
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1980) • _Our results are at the middle and right of Figure 1. 
We found that pfbgrammed instruct ionraised student test 
scores by 0,08 standard deviations in the typical^ secondary 
school. application^ and by 0.25 standard deviations in the 
typical college appl icat ion . 

It would be easy enough to dismiss programined 
instruction on the basis of these results. They seem^to 
show that programmed instruction has trivial effects, in 
elementary and secondary schools and small ones at the 
college level. But there is more to programmed instruct ion 
than this . - As_Figure 2 shows, each of the three meta- 
analyses demonstr-ated that programmed instruction was more 
effective in recent studies than in earlier qnes. 
Programmed instruct ion had a poor record in its "earlier 
years. It has a relatively good record in recent years. 



insert Figure 2 about here 



This is certainly a set of f indings that needs an 
explanation. _It_may be that the art of programming has 
improved in subtle. \f/ays in recent , years . It may be that 
evaluation methodology has improved. Or perhaps the early 
years gave teachers a sense of what programmed instruct ion_ 
can and cannot do^ and they now use it more selectively^ I_ 
do not know which of these explanat ions is the right one_. _ I 
feel confident, however, that meta-analysis will eventually 
help us choose between such competing explanations. 



Insert Figure 3 about here 



Individual i zed Systems 

The record . of effect iveness of individualized systems 
of instruction is equally p»erplexing . Figure 3 gives the 
overall meta-a^ialyt ic results. The results at the left on 
the figure are from Hartley's (1978) meta-analysis on 
elementary and secondary school findings; the results _in the 
center are from our analysis of secondary school findings _ 
(Bangert, Kulik, & Kulik, 1983); and the results at the left 
are from our_analysis of college f indings (J . Kulik, Kulik, 
& Cohen, 1979). The f igure shows that individualized, 
systems produce about the same results as convent ional. 
teaching. at the elementary and secondary levels. At the 
college level, however, individualized systems are 
strikingly ef f ect ive . 

What can account for such results? Again, we don't 
know for sure, but it's possible to speculate. 
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1. it . may be that .college learners have the cognitive 
ski lis _ recjuired by individualized instruct ion and that 
precbllege learners lack these skills.. To profit from 
individualized teaching^ learners need to be able to 
pace themselves, make choices, and work independently. 
College students are better prepared than precollege 
learners for such tasks. 

2. A second possible explanation is somewhat more complex i 
Individualized instruction at the college level has been 
greatly influenced by the work of psychologist Fred 
Keller and his colleagues^ Unlike the individualized 
systems developed for elementary and secondary schools , 
Keller • s system places great emphas is ^ on the ^ social 
support provided by peer tutors . it is possible that 
individualized systems have been successful at the ^ _ 
college level because of this unique emphasis on social 
support. 

3. A third possibility must also be invest igated. The 
studies used in the precollege _and secondary meta- 
analyses almost all came from dissertations^ whereas the 
studies at the college, level almost all came from 
journals and books. _The results from the different 
levels of_ education look more similar when they are 
reported by publication source (Figure 4). 



Insert Figure 4 about here 



It is impossible for me to choose right now between 
these three explanations . I have a hunch about which of^the 
explanations is the correct one, but i have not yet carried 
out the analyses needed to prove of disprove my hunch_. _ I 
feel confident , however, that future meta-analyses vill 
clear up the picture. 



Insert Figure 5 about here 



Computer-Rased Ed ucation 

In contrast to these findings on programmed and 

individualized instruction^ findings on computer-based 

instruction are. relat iyely straightforward (Hartley, 1977; 
C.-L. Kulik^ Kulik, & Bangert-Drowns , 1984; J. Kulik, 
Bangert,_St Williams 1983; J. Kulik, Kulik, & Cohen, 1980). 

Results from four different meta-analyses are rather 

consistent in showing that computer-kjased education has _reai 
potential as a tool in improving student achievement (Figure 
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5). The average effect of computer-based teaching was to 
raise student achievement by approximately 0 . 4 standard 
deviations, or from the BOth to the 55th percentile* 

Results from computer-based teaching appeared to be 
slightly stronger at the lover level. of instruct ion than at 
the higher levels. That is perhaps because programs _ _ 
developed so far exploit the capac it ies of computers most ^ 
adapted to lower level learning: their patience in drilling 

and tutorina students and their capacity to respond 

immediateiy^and appropriately to student answers. The 

effectiveness of the computer in college teaching may ^ 

improve when other capacities of computers are exploited m 
educat ion . 

Cbn clus ion 



Both primary _evaluat ion studies and meta-analyses are 
increasing _our understanding of instruct ional technology— -an 
important force that is now transforming education. Such 
studies are not only establishing a baseline for evaluating 
further developments^ but they are also point ing to areas 
where further developmental work needs to be done. 
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Figure 1 
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HETfl-flNHLTSES ON PROGRRHHED INSTRUCTIQN 
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METfl-flNnLYSES ON INDIVIDUALIZED SYSTEMS 
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Figure 3 
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METR-flNHLYSES QN i N0 J V I DUAL I ZED SYSTEMS 
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Figure 4 
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